
nternet Archive 



20 years / 20 petabytes 




Internet Archive 




Internet Archive is a non profit 
library of millions of free books, 
movies, software, music, and more, 
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Advanced Search 
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Lots of hard drives 
56.36 petabytes of disk space 
21 .93 petabytes of archival content 
Stored redundantly in different physical 
locations 




DIY storage cluster 



Archival on-disk format 
Efficient (cost + energy) 





Mffnet Arcliive Datacenter 




Sun 

microsystems^ 



' °^<^o,ani Java 




pload 



8M Texts 
2.4M Audio recordings 
140K Concerts 
1 .9M Videos 
1 0OK Software items 
and your stuff! 






S3-compatible API 
Metadata API 

^pip install internetarchive 



https : //archive. org/help/about s3.txt 




Using lA S3 to back distributed art projects 




ayback Machine 



Started in 1 996 with Alexa crawls 
Updated in realtime at 2000 uris/sec 
475,951 ,398,000 captures 
1 0.5 petabytes of WARC data 
IntI Internet Preservation Consortium 
Open source on github.com/internetarchive 



Realtime Wayback 
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De eted within two hours: 



hnp://vk.coni/strelkov_info 



554 captures 

|jn 11 - 28 Apr 11 
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Cdoakm ot Crpe/iKona Mropn MnaHOSMMa 

17.07.2014 17:50 (mck) Coo6»«eHMe ot ononMeHMn. 

"B paiioHe Tope3a tcwiwco hto c£>w]tn cawoner Ah-26, oantKrcat 

rae-To 3a luaKTOM "nporpecc". 

npeaynpotyia/iM Me - hc /leraib b "MaiucM He6e". 

A DOT n BMfleo-nonTBep)t<aenMe OMepeaMoro "nTHMKonaaa". 

ITrnHKa yna/ia 3a TeppMKOH, xwiofi cetaop mc 3aLienki/ia. 

MnpHwe ntOAH nocrpaoa/iki. 

A TatoKe eiue ecTb MH(t>opMauM$) o btopom c6mtom caMonere, 
Dpooe 6bi Cy." 





Realtime Wayback - Save Page Now 



Save Page Now 



http:// 



SAVE PAGE 



Capture a web page as it appears now 
for use as a trusted citation in the future. 

Only available for sites that allow crawlers. 



Save Page Now 
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http://blog.uber.com/ridesofglory 



13 captures 

Aug 14-27 Ma- 



o 
o 



Comic-Con 



Valentine's! 



Co 
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day of year 

"Brief overnight weekend stays" dropped precipitously during the 
days surrounding Valentine's Day. The patterns are fascinating. The 



J. 



NOV 
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ayback Under the Hood 



10.5 petabytes of WARC data 

2nd Level CDX Index: 20TB compressed 



1st level: 13GB. Fits in core. 



ayback APIs 





Availability and CDX APIs: build your own 
tools on top of Wayback 
Open source tools to read/write WARCs 
WARC Support in wget: 

wget --ware-file 
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Books 




2.5 million books with Scribe bookscanner 
1 000 books/day 

35 scanning centers on 5 continents 
800 million images 

Open-sourced gphoto Canon 1 D/5D drivers 



Books 




Books 



A general history of birds 
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Tlic; oiih desrri|iliuii I riiid, ^wkw bv riirskal, iiR-iitiuiis lliut 
fill' bini is w bully ut' a ^ix'eii ruluur: the lliruul yt-ltutv ; Hit' bill 
litai-k anil ratht-T strait ; tongue bii]<;iitatMl ; tail cunsisting of twelv« 
f«Rtli<Ms, un<l fveii at tlie end ; kjrs flvsli -colour. 

nliiibitii R}[}'|it ; tliv Arabian nanit; is C'liiuldicjr. Tlit! j-uuiif; art; 
«olil in tile inontli of iwwv. 



m-l C. M. Oth. \. 270. 3. 1 



i^ngtli six inches and alialt'. Pluuia^ tbe same with tht^ utlicr^, 
excvpling that llie streak on sid<- of tlie head, and tbe band 
on IIk- tltitinl art.' blue ; the two nii«ldle tail fuithcrs do not cxi'ml 
tbi- ollii:rK in length, but art; narrow tlie » huk- nay, and black. It 
is \*:x\ probable, thai llit bird might have tost the two old feallierK, 
and had bien supplieti with two new ones, not ytt arrived at Ibe 
full gruwih; in which ca.'x*', ihi^i hinl would Ik- very little shurler 
llian llie first descrilied, at llie e\c«M of leiiglh in that arcMc merely 
from the two middle feallier«. 



. p laa ImA On. 1 1 • 

This vsrtely haM a yellow forehead, and u deep blue throat; tbe 
rent of the plumage as in Ihe firsl <ii ■^crilM'd . 

This is a ittmmon 5|>eeies in Indin. where they are often Been 
flying about, and are said In be fond of plantains. 
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TV NEWS ARCHIVE obama 



About your Search 




CANDIDATE 

Tom Corbett (con) 2 
Tom Wolf (con) 2 
John Kane (pro) 1 
Mike Fitzpatrick (pro) 1 
Tom Corbett (pro) 1 
Tom McGarrigle (con) 1 

SPONSOR 

Tom Corbett for Governor 3 
Kane for Senate 2 

Philadelphia Federation of Teachers 2 
Fitzpatrick for Congress ' 



Search Results 0 to 49 of about 389,622 (some duplicate have been removed) 
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Ar^^ica Torioht 

Al Jazeera America 

Jan 30. 2014 4:00am EST 


The Rachel Maddow Show 
MSNBC 

Aor21.2015 1«JamPDT 


president who revoked his passport 
president ^^R. Wednesday on the hill 
the senat^ntenigence committee warned 
that he is a threat to this nation's 


president ^^^3 'fid not miss more 
than half his 



Guide Me 



GRID SCAN ' 




FQXN«ws 

Jan2.201011«k)niESr 



^ for the country, and for president 

. amid high unemployment, will 

ne economic recovery continue? and 
will obama's declining approval ratings, 




Television 




Internet Archive 



chive-It 



Audio Books 
Bitcoin ATIVI 
Bittorrent seeds 
Bookmobile 
Book Reader 
Book Scanning 
CD archiving 
Community Wireless 



redit Union 



ISY/ Print 



led 



ed Web 



ation 



Foundation Housing 
Heritrix crawler 
IIPC 

JSMESS emulator 
Lending Library 
Listening Rooms 
Live Music Archive 
LP digitization 
Metadata API 
Microfilm scanning 
Microfiche 
Netlabels 

Newspaper digitization 



OPDS 

Open Content Alliance 
Open Library 
Petabox 
Physical Archive 
Reader Privacy 
Researcher Access 
S3-like API 
Scribe and TTScribe 
Software preservation 
Television Archive 
V2 site 

VHS digitization 
Wayback Machine 



uestions? 



• developers.archive.org 

• we love volunteers! 

• we are hiring engineers! 
arcliive.org/jobs 




